Papers with model size tradeoffs
Combining Compressions for Multiplicative Size Scaling on Natural Language Tasks (2022.coling-1)
Copied to clipboard
| Challenge: | Quantization, knowledge distillation, and magnitude pruning are among the most popular methods for neural network compression in NLP. |
| Approach: | They compare accuracy vs. model size tradeoffs using quantization and distillation methods . they find that pruning provides greater benefit than quantization . |
| Outcome: | The proposed methods reduce model size and can accelerate inference, but their relative benefit and combinatorial interactions have not been rigorously studied. |